Skeleton-based action recognition via spatial and temporal transformer networks

Skeleton-based Human Activity Recognition has achieved great interest in recent years as skeleton data demonstrated being robust to illumination changes, body scales, dynamic camera views, and complex background. In particular, Spatial-Temporal Graph Convolutional Networks (ST-GCN) be effective learning both spatial temporal dependencies on non-Euclidean such graphs. Nevertheless, an encoding of the latent information underlying 3D is still open problem, especially when it comes extracting from joint motion patterns their correlations. this work, we propose a novel Transformer network (ST-TR) which models between joints using self-attention operator. our ST-TR model, Spatial Self-Attention module (SSA) used understand intra-frame interactions different parts, Temporal (TSA) model inter-frame The two are combined two-stream network, whose performance evaluated three large-scale datasets, NTU-RGB+D 60, 120, Kinetics Skeleton 400, consistently improving backbone results. Compared with methods that use same input data, proposed achieves state-of-the-art all datasets joints' coordinates input, results on-par adding bones information.